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ABSTRACT 

Brain-Computer Interfaces (BCI) are communication sys¬ 
tems conveying messages through brain activity only. This 
paper elaborates on the suitability of BCI for 3D Video 
Games (VG). Thus, we first review some recent BCI-based 
3D VG. We then discuss the limitations of current BCI tech¬ 
nology, those being mainly related to usability and perfor¬ 
mances. Finally, we report on some areas in which BCI 
could be useful for 3D VG despite their limitations. More 
precisely, BCI could be useful as an additional control chan¬ 
nel, to send commands that cannot be intuitively sent with 
other devices. BCI could also be used for mental state mon¬ 
itoring either 1) during the game, in order to make adaptive 
and dynamic video games or 2) during the game creation 
in order to maximizes some measures of game quality that 
could be derived from a tester’s mental state. 

1. INTRODUCTION 

For long, keyboards, mice and pads have been the devices of 
choice to play Video Games (VG). However, since the arrival 
of the Nintendo Wiimote® or the Microsoft Kinect®, the 
hegemony of these old devices is fading away, and full body 
interaction with 3D VG is becoming the new trend. In this 
context, the VG community is looking for new interaction 
devices that can make this full body interaction more natu¬ 
ral, immersive and enjoyable for the player. Brain-Computer 
Interfaces (BCI) could be that new device. Indeed, BCI are 
devices that enable a user to convey messages to a computer 
thanks to brain activity only, this activity being generally 
measured using ElectroEncephaloGraphy (EEG) [18]. 

Recent research results have shown that BCI can indeed 
be used to interact with 3D VG and Virtual Environments 
(VE) [8] [14]. However these results are laboratory-controlled 
results and as such more a proof-of-concept than a real life 
application. One may therefore wonder how useful BCI can 
really be in the context of 3D VG. In this paper we address 
this specific question. In Section 2, we briefly review some 
recent results in BCI-based 3D VG. Then, in Section 3, we 


discuss the limitations of current BCI technology. Finally, 
Section 4 discusses some promising areas in which BCI might 
prove particularly useful for 3D VG in the future. 

2. RECENT RESULTS 

During the past 10 years, a number of prototypes of BCI- 
based 3D VG have been proposed (for reviews about BCI 
and all kinds of games, see [14] [8]). Some of these prototypes 
enable the player to select 3D objects in the game simply 
by paying attention to these objects. In this case, a specific 
visual stimulus is displayed on overlay of each object. It 
is indeed possible to find out whether the player is paying 
attention to a given object by analyzing the player’s EEG 
signals in response to the corresponding stimulus [1] [7]. 

However, most existing prototypes of BCI-based 3D VG use 
brain activity to perform navigation tasks in the VE (see 
e.g., [9][20][19][3]). Although these works prove that explor¬ 
ing a VE by brain activity is possible, the number of de¬ 
grees of freedom provided to the user is rather limited. In¬ 
deed, these works offer only 1 or 2 mental commands to the 
player (e.g., turning left or right by imagining left or right 
hand movements, respectively). Moreover, most of these 
BCI were synchronous, meaning that the player could only 
send mental commands at specific time periods, imposed by 
the system, which is neither natural nor convenient. Re¬ 
cently, more advanced BCI systems have been proposed to 
navigate VE [17] [13]. These BCI provide 3 different men¬ 
tal commands to the user, and were asynchronous BCI, i.e., 
the player could send mental commands at anytime, at will. 
Moreover, [13] showed that by using appropriate interaction 
techniques, only 3 mental commands were necessary to ex¬ 
plore large and complex 3D VE (see also Figure 1). 

3. HYPE: LIMITATIONS OF BCI 

Even though such recent results are rather impressive, these 
BCI-based 3D VG remain simple prototypes that are not 
yet suitable for practical applications and widespread use. 
In particular, most if not all of these prototypes were as¬ 
sessed in laboratory conditions, i.e., in a carefully controlled 
environment, very different from an actual gaming context. 
More precisely, most current BCI were used in artificially 
quiet environments with artificially quiet subjects. There¬ 
fore, in a real gaming context, we would expect BCI per¬ 
formances - which are already rather low (e.g., the recog¬ 
nition of two mental states only very rarely reaches 100% 
accuracy)- to get even worse. Although there were some ex¬ 
ceptions with evaluations in close-to-real-life environments, 




Figure 1: BCI-based navigation in a complex VE. 


e.g., “Alpha WoW” [14] and “Use-the-force” [11], there is still 
a number of limitations impeding BCI use for actual 3D VG. 

First, BCI users so far always have to sit, since body mo¬ 
tions are known to generate various electrical artefacts that 
pollute EEG signals [4]. Unfortunately, during full body in¬ 
teraction with a 3D VG, the player will naturally be doing 
a lot of body motions which may jeopardize BCI use. 

Second, using a BCI requires a lengthy and tedious cali¬ 
bration step. This step consists in collecting examples of 
EEG signals from the player in order to adapt the BCI to 
his/her specific brain activity. There is indeed a large inter¬ 
subject variability. The length of this calibration step ranges 
roughly from 5 to 20 minutes for the best BCI, depending on 
the mental states used to drive the interface [2]. Since most 
players probably want to use their game device immediately, 
such a calibration step would probably discourage many of 
them from using a BCI for gaming. 

Third, current EEG sensors are gel-based, meaning that the 
subject needs to put gel in his hair below the EEG electrodes 
in order to use the BCI. Naturally, very few players would 
accept such a drawback if they play on a regular basis. 

Fourth, for roughly 20% of BCI users, the recognition per¬ 
formances they achieve are not good enough to enable any 
form of control with this BCI [18]. Although the causes of 
this so-called “BCI illiteracy” are not well understood, in 
practice this may be a problem as a game device that 20% 
of the players cannot use may be quickly abandoned. 

Fifth, the number of mental states that can be recognized 
by a BCI is currently very limited. For instance, most 
BCI based on imagined movements can only discriminate 
between a few tasks (e.g., imagining left hand movements 
versus imagining foot movements). Most probably, a game 
designer or a player would like the BCI to recognize a much 
larger variety of mental states, ideally tailored to the game 
at hand. Unfortunately, only a very limited number of men¬ 
tal states have been explored so far [18]. 

Finally, in terms of performances, a BCI is far below all ex¬ 
isting game devices. More precisely, a BCI is much slower 
and much more prone to errors (e.g., wrongly identifying a 
given mental state instead of another one) than other de¬ 
vices. This makes the use of a BCI to send control com¬ 


mands rather inconvenient. Actually, the very limited per¬ 
formances of an EEG-based BCI make its use as the main 
control device of a 3D VG rather unlikely, even in the future. 

4. HOPE: PROMISING PERSPECTIVES 

Even though the previous section may seem very pessimistic 
about the future of BCI for 3D VG, there are also many rea¬ 
sons to be optimistic. For instance, many of the limitations 
mentioned above are currently being addressed by the BCI 
community and improvements can be expected in the rela¬ 
tively short term. More precisely, some preliminary results 
suggested that BCI could be used by a mobile user [12], 
although this makes the recognition performances slightly 
lower. Some efforts have also be made to reduce or suppress 
the calibration time with a minimal (but, unfortunately not 
null) performance loss [10] [5]. The design and use of dry 
EEG sensors (i.e., without gel) is also being explored and 
validated, although this again leads to a slight performance 
loss as compared to gel-based sensors [15]. Finally, a new 
approach to BCI calibration has been proposed in order to 
reduce the number of BCI illiterates [21]. 

Unfortunately, one cannot expect much of EEG due to its 
very nature: noisy, non-stationnary signals of poor spatial 
resolution (EEG measures the synchronous activity of mil¬ 
lions of neurons whereas we are ideally interested in their 
individual activity). Therefore, EEG-based BCI may always 
have modest performance which may prevent them from be¬ 
ing the primary control device for VG. Nevertheless, BCI 
may be suitable for 3D VG in other ways. They could be 
used as an additional control channel, supplementing a game 
pad or a Kinect, for instance. While the low performances 
and speed of BCI would prevent them from being used to 
send critical commands, it can be perfect to send optional 
commands. Such a command could be a super power that is 
not needed to complete the game but can make things easier 
for the player if he manages to control the BCI. In this sense, 
as Nijholt put it and explored in “AlphaWoW”, the short¬ 
comings of BCI are turned into a challenge for the player 
[14]. In the case of immersive 3D VG, a BCI could also be 
used to increase this immersion by being used to send com¬ 
mands that cannot be intuitively issued with other devices 
such as a Wiimote or a Kinect. For instance, if the player’s 
character in the game is a monkey, a full body interaction 
technique cannot enable him to perform tail movements in 
the game as a typical player does not have any. However, 
imagining a tail movement is something relatively easy for 
the user, that a BCI could potentially detect. Whether such 
a mental state could actually be detected in EEG signals is 
still an open question but even so, using a BCI in this way 
could again benefit to the player’s immersive feeling in the 
3D environment and thus to the fun of the game. 

Another area in which BCI could be a promising tool for 
video games is mental state monitoring. Rather than using 
a BCI in an active way to voluntarily send control com¬ 
mands to the game, the BCI could be used instead in a 
passive way (in this case the BCI is denoted as a “passive 
BCI”, see [6][22][18] for reviews), in order to dynamically 
adapt the content of the game to the current mental state of 
the user 1 . For instance, if a real-time analysis of the player’s 

1 These aspects are currently explored in the OpenViBE2 




EEG signals reveals that the player is bored, the game dif¬ 
ficulty might be increased. On the contrary, if the mental 
workload of the user is too high, then the VE might be sim¬ 
plified to make the interaction and gaming easier or at least 
more intuitive. In these case, the limited performance of the 
BCI is not an issue, since a correct detection of the mental 
state is not required to complete the game, does not need 
to be fast but can possibly make the game experience more 
enjoyable. A passive BCI can also be useful for game design 
and evaluation. Indeed, while some players might feel un¬ 
comfortable wearing an EEG cap, it is perfectly acceptable 
to ask a 3D VG tester to wear one while assessing the game 
during its creation. This would enable the game designers to 
finely assess which parts of the game should be improved or 
removed in order to maximize the player’s enjoyment based 
on the tester’s brain responses to the different game events. 
Similarly, this would enable the designers to make the in¬ 
teraction technique used to control the game as intuitive as 
possible, based again of the tester’s mental states while us¬ 
ing it. There is currently only little research in the area 
of mental state monitoring, so it remains difficult to predict 
what kind of mental states can be recognized in EEG signals 
and how well they can be recognized. However, preliminary 
results are encouraging and these topics are getting increas¬ 
ingly popular so we may expect some interesting and useful 
progresses in a relatively near future [6] [22] [18]. 

Finally, it should be mentioned that, while EEG signals have 
many limitations, many of these limitations can be solved by 
using sensors implanted within the brain [18]. Naturally, this 
comes with the strong limitation of being highly invasive. 
However, one may except that, in the future, some hardcore 
gamers would ask for such invasive sensors in order to be 
able to play more advanced and innovative VG. 

5. CONCLUSION 

In this paper, we elaborated on whether EEG-based BCI 
were suitable devices for 3D video games. More particu¬ 
larly, we first provided a brief overview of recent 3D video 
games controlled by a BCI. Then we discussed the limita¬ 
tions of current BCI technologies, these limitations being 
mainly related to usability (inconvenient need for gel-based 
electrodes and long calibration times) and performances (low 
command production rate and relatively high recognition er¬ 
ror rate, especially in a gaming context). Finally, we pro¬ 
posed a number of areas in which BCI could still be useful 
for 3D games despite the limitations identified. More pre¬ 
cisely, BCI could be useful as an additional control channel, 
to send commands that cannot be intuitively sent with other 
devices. Moreover, BCI could also be useful to design adap¬ 
tive video games that dynamically change depending on the 
player’s mental states. The BCI could also be used dur¬ 
ing the creation of the game itself, in order to design a game 
that maximizes some measures of game quality derived from 
a tester’s mental state (e.g., flow, presence, etc.). In conclu¬ 
sion, we believe that BCI-based 3D VG is a very promising 
topic that needs to be further explored. The availability of a 
free and open-source software platform to create such BCI- 
based VG [16] should encourage the gaming community to 
join the RVD activities in this area. 
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